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Abstract. The paper introduces a novel iterative method that assigns a rep- 
utation to n + m items: n raters and m objects. Each rater evaluates a subset 
f^^ of objects leading to a n X m rating matrix with a certain sparsity pattern. 

{"* **) From this rating matrix we give a nonlinear formula to define the reputation 

f"*^ of raters and objects. We also provide an iterative algorithm that superlin- 

fsj early converges to the unique vector of reputations and this for any rating 

matrix. In contrast to classical outliers detection, no evaluation is discarded 
in this method but each one is taken into account with different weights for 
the reputation of the objects. The complexity of one iteration step is linear in 
S , the number of evaluations, making our algorithm efficient for large data set. 

Experiments show good robustness of the reputation of the objects against 
cheaters and spammers and good detection properties of cheaters and spam- 
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1. Introduction 



There is an important growth of sites on the World Wide Web where users play 

a crucial role: they provide trust ratings to objects or even to other raters. Such 

sites may be commercial, where buyers evaluate sellers or articles (Ebay, Amazone, 

etc.), or they may be opinion sites, where users evaluate objects (Epinions, Tailrank, 

MovieLens, etc.). But websites are not the only place where we can find ratings be- 

vq tween users and items: the simple fact to link to another webpage is considered by 

^V search engines as a positive evaluation (Google, Yahoo, etc.). Therefore the good 

working of auction systems, opinion websites, search engines, etc. depends directly 

^__l on the reliability of their raters and on the treatment of all the data. Trust and rep- 

t-H utation in the electronic market gives a necessary transparency to their users. For 

example, in 1970, Akerloff [7] pointed out the information asymmetry between the 

^-^ buyers and the sellers in the market for lemons. The former had more information 

^- than the latter, making hard trusting trading relationships. From what precedes, 

'ku two questions naturally arises: 

- What should be the reputation of the evaluated items? 

- How can we measure the reliability of the raters ? 

We will distinguish the reputation, that is what is generally said or believed about 
a person's or thing's character or standing, and the reliability, that is the subjec- 
tive probability by which one expects that a rater gives an evaluation on which its 
welfare depends. Let us remark that many technics only calculate the reputations 
of items. Sometimes reputation and reliability have the same value as it is the case 
in eigenvector based technic where the reputation of any individual depends on the 
reputations of his raters [51 IE]- In these methods, they construct a stochastic ma- 
trix from the network and the ratings, then the eigenvector of that matrix gives the 
reputations. Another part of the literature concerns the propagation of trust (and 
distrust) [HI Uni El E] where they define trust metrics between pairs of individuals 
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(A, B) looking at the possible paths linking A with B. So reputations depend on 
the point of view of the user and these methods differ from ours that assigns one 
global reputation for each item. 

Our method weights the evaluations of the raters. A small weight is a natu- 
ral way to tackle the problem of attackers in reputation systems. Therefore, the 
method gives two values for a user: his reputation depending on his received eval- 
uations and his weight that influence the impact of his given evaluations. The 
algorithm is based on an iterative refinement that is guarantied to converge to a 
reputation score and a reliability score for each item: at each step the reliability of 
a rater is calculated according to some distance between his given evaluations and 
the reputations of the items he evaluates. This distance is interpreted as the belief 
divergence. Typically, a rater diverging to much from the group will be distrusted 
after convergence. The same definition of distance appears in [TJ |SJ [3J and is used 
for the same issue. In [!■, the function that determines the weights is different. This 
difference makes their algorithm sensitive to initial conditions without any guaranty 
of convergence. Moreover, they are in the less general case where it is supposed that 
every rater evaluates all items. In [2 , they want to tackle the problem of spam- 
mers in collaborative filtering where previous evaluations are used to predict future 
evaluations. Again the same definition of distance allows to penalize the divergent 
raters. Even though the function that assigns the weights for the evaluations is the 
same, there is no iterative procedure but only a simple step is applied. In [3J, they 
use another function to determine the weights: the log-likelihoods, but again only a 
simple step is applied. We show in section [3] the advantage to apply more than one 
step in the iterative filtering. Indeed, each step separates a little more the outliers. 

Let us remark that beside the refinement process of the reputations and the 
outlier detection given by our procedure, other applications can take advantage of 
these data. For example, [2] want to remove spammers to improve collaborative 
filtering. Similarly in [4 , they propose a framework to take into account the dif- 
ferent qualities of ratings for collaborative filtering. Hence they weight each rating 
according to its reliability, these weights can be those obtained by the iterative 
filtering we described. 

In the sequel, we first explain in section [2] how the reputation vector for the 
objects and the weights for the evaluation are built. Moreover, we develop the 
algorithm Reputation that calculates these values, and we explain its interpretation 
and its properties of convergence. Then in section [31 our experiments test the 
robustness of our method against attackers and show several iterations on graphics. 
Finally in section [4] we point out possible extensions and experiments for our 
method. 
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2. The iterative method 

Before to develop the model and the algorithm, we introduce the main notations 
in the following tabular. 



Notations 


Definitions 


n, TO, TTbi 
E 

A 

T 

t 

r 

1 

V. . . 

y. . . 


# of raters^ of items, 

# of items evaluated by i. 


The n x m rating matrix: 
Eij is the evaluation given 
by rater i to item j 


The n x m adjacency matrix: 
Aij — 1 if rater i evaluates j, 
otherwise A^ = (and E^ = 0) 


The n x m trust matrix of evaluations 
The nxl trust vector of raters 
The to x 1 reputation vector of items 
The n x 1 or m x 1 vector of ones 


Sum over the set {i|^4ij = 1} 
Sum over the set {j|Ay = 1} 



Without loss of generality, we will consider ratings in the interval [0,1], i.e. E £ 
[0, l] nxm and therefore the reputation vector r will belong to [0, 1]™. Moreover, 
the trust matrix T and the trust vector t are nonnegative, i.e. the entries of T and 
t are nonnegative. 

2.1. The model. As already said in the introduction, the reputations of the items 
essentially depend on the evaluations they receive. These latter are weighted ac- 
cording to their reliability. In that way, the reputation of item j G {1, . . . , to} is 
obtained by taking the weighted sum of its evaluations, i.e. 



(i) n = Y, w »Ei3, E w v = 1 

i, i — >j i, i — vj 

And we define the matrix W from the trust matrix T in the following way: 



(2) 



W, 



T 



*7 
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■ T kj 



1. 



J = 1 • • 



In that manner, evaluations with a higher trust value are taken into account more 
for the reputation vector. Now the important role is played by the trust matrix T, 
its definition is given in the next section. 



2.2. The trust matrix. Let us describe the trust matrix that assigns a measure 
of confidence to each rating. The inputs of the trust matrix are the rating matrix 
E and the reputation vector r. Formally, we define the belief divergence of rater 
ie {1, . . . , n\ as the estimated variance of the i th row of E: 



(3) 



, n} as the estimated variance of the i 
1 



») 



E 



(Eij-rtf 



where TOj is the number of items evaluated by i. That definition is somewhat similar 
to the one proposed in [5] where d is used to penalize those raters that have an high 
belief divergence. The resulting trust matrix is 



(4) 



■*■*! ~ c j 



4 CRISTOBALD DE KERCHOVE AND PAUL VAN DOOREN 

Trust Weight 

for each rating T — > W for each rating 

r i 

Belief divergence d < — r Reputation 

for each rater for each item 

Figure 1 . Cycle of one iteration of the algorithm. 

for any evaluation from i to j. The parameters Cj are chosen such that the entries of 
T are nonnegative. Moreover Cj are discriminating in the sense that they influence 
the ratios Tij/T^j for i,k £ {1, . . . , n} and j £ {1, . . . , m}. Typically, the smaller 
Cj, the more spammers evaluating object j are penalized. In order to have a trust 
value for each raters, we also define the trust vector t: 

\& ) t% U"max O'i, 

where d mnr is the maximum of the elements of the vector d. 



2.3. The algorithm. From equations (Ufl5|, we can derive the algorithm [r t] = 
Reputation(E,A, c) that takes as inputs a rating matrix E, an adjacency matrix 
A and a m x 1 vector c of parameters. Then it iteratively calculates the reputation 
vectors r and the trust matrix T. Eventually, the algorithm gives the reputation 
and the trust vector. The description is given in four steps corresponding to the 
initialization, two updates for r and T and the calculation of the trust vector t. 

(i) Initialization of matrix T: every rater is evenly trusted, i.e. T^- = 1 for 
i = 1 , . . . , n and j = 1 , . . . , m. 

(ii) The matrix of weights and the reputation vector arc calculated from T. For 
i = 1 , . . . , n and j = 1 , . . . , m: 

Wa = Tl3 , ri = J2 w v E a- 

(iii) The belief divergence and the new trust matrix are calculated from r. For 
i = 1, . . . , n and j = 1, . . . , m: 



- £ (E i: 



di= — 2^ (Eij -rj) , Tij = cj - d, 

Tfli . . 

3, *^J 

If the i th row of T is zero, then replace it by a row of ones. 
Repeat steps (ii) and (iii) until convergence. 

(iv) The trust vector t is given by maxt d^ — di. 

Let us remind that the input parameters c used in step (iii) are chosen sufficiently 
large such that the entries of T are nonnegative at each iteration. 

One iteration of the algorithm Reputation is schematized in figure [l] A slight 
modification allows dynamical evaluations. In that case, the rating matrix E 
changes at each iteration, i.e. E[k] with k = 1,2, •••. A direct way to update 
r, T and t, is given when the first step takes as initial vector the previous trust 
matrix, and steps (ii) and (iii) are not repeat until convergence, but a certain 
number of times. 
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2.4. Interpretation of the solution. The algorithm in section 2.3 converges to 



the unique solution of equations (Ij4l, see next section. Let r* represents that 



solution^ and for the sake of simplicity, let parameters Cj be equals to a same 
constant c . Then r* is the maximizer of the following scalar function: i\) : [0, l] m — > 
R with 



(6) Mr) = £ E T 'i 



2 
ij 
1 3, i^>j 



(7) = E 



1 \ 'j, i^j J 



It is indeed enough to observe that grad ip(r*) = 0. In other words, r* maximizes 
the Frobenius norm of the sparse trust matrix T. Maximizing such a norm roughly 
means that some total degree of confidence over the raters is maximized. More 
formally, let assume that the entries Ey are i.i.d.^ N(rj,a 2 ). In that case, the 
degree of confidence we can have in evaluation Eij is given by 



logPr(JS„ ;\ rj ) = est - — g(^y - Tj f 



1 
2V 2 



and by summing the evaluations of i and choosing the appropriate constant, we 

obtain the relation 

2ct 2 
^■^— E logPr(%h). 

k, k — yj 

In other words, the trust matrix is the normalized sum of the degrees of confidence 

we have in the evaluations of the raters. Then the maximizer of the Frobenius norm 

of Tis 

2 



^E^(E l0 ^^|r,)j 




r = arg max 

r£[0 " 

Another writing of the function ip leads to 

n n 

<Kr) = -2 C0 £ E (^ - ^) 2 + E - ! E @v - r i? 

and the maximizer of the first expression is simply the average of the evaluations for 
each item j, i.e. J2i i^j ^ij/\h * ~* J\-> an d the maximizer of the second expression 
necessarily belongs to the border of the hyper cube, i.e. {0, l} m . The solution r* 
is then a compromise between both terms in which the parameters cq plays the 
role of a weighting factor. For large c, the algorithm will give the average of the 
evaluations for each item. 

2.5. Properties of convergence. In this section we analyze the convergence of 



Reputation given in section 2.3 and its rate of convergence. For the sake of clear- 
ness, we restrict ourselves to the main and important steps in the proof avoiding 
the technical points. 

Theorem 1. For Cj chosen such that the matrix T is nonnegative at each step, the 
iteration in the algorithm Reputation(E, A, c) converges to the unique vector r* . 



the corresponding matrices and vectors W* , T* , d* and t* follow from 
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Figure 2. X-Axis: # of iterations. Y-Axis: the euclidian norm 
of the error, i.e. \\r[k] — r*\\ 2 - The graphic was obtained from a 
943 x 1682 sparse rating matrix representing 10 5 evaluations. 

Proof. (Only a sketch) . For the sake of simplicity, we let parameters Cj be equals 
to a same constant cq. We first prove the unicity of r* and then we prove the 
convergence result. 

It can be shown that the scalar function ip : [0, l] m — » R : r i— > ip(r) defined in 
(fn) is continuous and quasiconcave. It has therefore a unique maximizer on [0, l] m 
that corresponds to the vector r* . 

Let rfe and rj. +1 be two successive iterations of r. It can be shown by simple 
developments that these two vectors are linked by the following expression: 



(8) 



rk+i = r k + a(r k ) ■ grad ip(r k ), 



where a(ffc) > (4co) _1 > and grad ip(rk) is the gradient of ip in r k pointing to 
the direction of greatest ascent. Therefore one iteration corresponds to take the 
direction of greatest ascent and make a step of length t k — a(r k ) ■ ||grad ^(rfc^h- 
Moreover, it can be shown that £ k is such that we have strict ascent: 

ip(r[k}) <ip(r[k+l]). 

Finally, £ k is also lower bounded by (4co) _1 ||grad T/>(r[fc])||2 and therefore the iter- 
ation on r monotonically converges to the maximizer of if) on [0, l] m . D 

Numerical experiments show a linear rate of convergence for the vector r. As 
shown in Figure [2] the logarithm of the error decreases linearly and stabilizes after 
20 steps. It is possible to speed up the rate of convergence by using a Newton 
method provided that we are close enough to r* . Then the rate of convergence 
becomes quadratic making our algorithm efficient for large data set. 

3. Experiments 

Our experiment concerns a data seijjof 100,000 evaluations given by 943 users 
on 1682 movies and raging from 1 to 5. Each user has rated at least 20 movies. 

In order to simulate the robustness of the algorithm Reputation, two types of 
behavior are analyzed in the sequel: first, raters that give random evaluations, and 
second, spammers that try to improve the reputation of their preferred item. 

3.1. Robustness against random raters. We added to the original data set 
237 raters evaluating randomly some items. In that manner, 20% of the raters give 
random evaluations. Let r* and r* be respectively the reputation vector before 



The MovieLens data set used in this paper was supplied by the GroupLens Research Project. 
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Figure 3. X-Axis: the sorted movies according to their reputa- 
tions before the addition of random raters. Y-Axis: their reputa- 
tions according to our algorithm (Top) and to the average (Bot- 
tom). 



and after the addition of the random raters. If the reputation vector is calculated 
according to Reputation, then the 1-norm difference between r* and f* is 

||r*-f*|| 1 = 182, 

if the reputation vector is the average of the evaluations for each item, then the 
1-norm difference between r* and f* increases: 

\\r* -f*||i = 259. 

Figure [3] illustrates this perturbation due to the addition of random raters. The 
reputations are better preserved when using Reputation. It turns out that the 
reputations given by Reputation take less into account the random users. More- 
over, one iteration of the algorithm gives poor information to trust the raters, it is 
indeed useful to wait until convergence, as seen in Figure HI 

3.2. Robustness against spammers. We now added to the original data set 237 
spammers giving always 1 except for their preferred movie, which they rated 5. Let 
r* and r* be respectively the reputation vector before and after the addition of 
the random raters. If the reputation vector is calculated according to Reputation, 
then the 1-norm difference between r* and f* is 



\r* -r*\U =267, 
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Figure 4. X-Axis: the trust values for the raters. Y-Axis: the 
density after one iteration (Top), after two iterations (Left), and 
after convergence (Right). In black: the random raters. In white: 
the original raters. In grey: both raters. 
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Figure 5. X-Axis: the sorted movies according to their reputa- 
tions before the addition of spammers. Y-Axis: their reputations 
according to our algorithm (Top) and to the average (Bottom). 



if the reputation vector is the average of the evaluations for each item, then the 
1-norm difference between r* and f* increases: 

||r*-r*||i=638. 

Figure [5] illustrates this perturbation due to the addition of spammers. The repu- 
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Figure 6. X-Axis: the trust values for the raters. Y-Axis: the 
density after one iteration (Top), after two iterations (Left), and 
after convergence (Right). In black: the spammers. In white: the 
original raters. In grey: both users. 

tations are again better preserved when using Reputation. Again the reputations 
given by Reputation take less into account the spammers. As previously, one iter- 
ation of the algorithm gives poor information to trust the raters, it is indeed useful 
to wait until convergence, as seen in Figure [6j 



4. Conclusion and Future Work 

Our method described in the paper allows us to efficiently refine reputations for 
evaluated objects from structured data. It is based on the trust we can have in 
the evaluations of the raters, and also in the raters themselves. The parameters 
Cj, introduced in equation HI make the method flexible, ranging from the average 
method, i.e. every rater is evenly trusted, until the discriminating method that 
takes Cj as small as possible. 

The experiments show interesting results of robustness even though the behavior 
of the added outliers is somewhat naive. The weights of spammers and random 
raters are low for the aggregation of the reputation vector. However, other behaviors 
could be analyzed. For example, clumsy raters could evaluate once correctly and 
once randomly or we can imagine a more complicated mix of behaviors. Typically, 
the weights of such raters will be between those of spammers and those of honest 
raters. Last but not least, the creative cheaters can use engineering to understand 
the working of the system. The way to proceed is simple: they need to evaluate 
correctly a group of item and then with that trust, they can rate some target items. 
In order to significantly change the reputation of these target items, they must have 
a number of coordinated evaluations larger than the one of honest raters. Therefore 
such cheaters can easily be disqualified by looking after coordinated ratings to one 
or several items. 



As said at the end of section 2.1 the trust matrix T is the important point for 
the model. We define it by 

ij — ^'j * ) 

for any evaluation from i to j. Hence, the trust we have in evaluation Tij decreases 
when the belief divergence rfj increases. Other decreasing functions with respect to 



d make sense. For instance, 



±ij — e 



and Ti, 



on some data sets. The second definition with 



{cj + di) may perform well 
gives the method described 



in [1 . However, the main difference with our definition lies in the uniqueness of 
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the solution. It turns out that the method in [Tj may have several solutions. On 
the other hand, these solutions can be of interest if they reflect for example two 
opinion trends. 

In section |2.4| the solution is interpreted as the maximizer of the Frobenius 
norm of T. It is possible to maximize other norms of T. Then there can be several 
maximizers and these maximizers will no more satisfy equation (nl , but a different 
one. 

We see that our method can be extended towards different directions. Our future 
work will address the interpretation and the convergence of these extensions. 
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